SE-Validation

Software Testing

• A very important activity to verify the software
• It is used to check if the software contains bugs that clients do not expect

Testing vs debugging

• Testing: check if the system has a bug

• Debugging: locate the position of a bug

• Verification

  • Check if the system meets the specification
  • Are we building the system properly?

• Validation

  • Check if the system meets users' requirements
  • Are we building the proper system?

Test Case Generation

• Black-box testing (Functional testing)
• White-box testing (Structural testing)
• Gray-box testing (Combination of both)

Black-Box Testing

• Functional testing
• Test cases are generated based on the requirement specification of software, rather than the source code
• Strategies for black-box testing include
  • Equivalence-Class Testing
    • Input domain D is divided into disjoint sub-domains according to some relation. Then from each sub-domain, a representative is chosen as a test case
  • Boundary Value Testing
    • This method involves selecting test case values that are near the boundaries of input domains
    • The number of test cases can be calculated as $4\times (n+1)$, where $n$ is the number of input variables.

White-Box Testing

• Structural testing
• Test cases are generated based on the ** of software
  • Statement testing
    • Every statement must be executed at least once
  • Branch testing
    • Every branch must be executed at least once
  • Path testing
    • Every path must be executed at least once

---

Other Testing Methods

• Transaction flow testing
• Transition testing
• Logic-based testing
• Fault-based testing
• Metamorphic testing

---

How to create test cases with black box testing and white box testing?

Black-Box Testing (Functional Testing):

• Test cases are generated based on the requirements specification of the software.
• Specifications should ideally be written in a notation that can be handled by computers.
• Strategies for generating test cases include:

White-Box Testing (Structural Testing):

• Test cases are generated based on the source code of the software.

• The source code is analyzed to find test cases that satisfy certain conditions.

• Automating test case generation is the most difficult job in automatic software testing

• That is why many testers use Random Testing Method

• Some tools are available in market to support boundary value testing.

What are the testing levels?

• Unit Testing: Testing a single program or a function.
• Integration Testing: Testing a module or an integration of several modules.
• System Testing: Testing the entire system.
• Regression Testing: Performed after fixing bugs, using the same test data as before to ensure bug fixes don't introduce more bugs. It's performed across different levels of the software development life cycle.

Testing Process

• Plan Test: This phase includes preparing test cases, writing a test plan, setting the testing environment, and selecting testing tools. A test plan specifies details like the module to test, testing strategies, tools, test cases, and expected output.
• Test Execution: Test cases are used as input for the program under test. The actual results from executing the test cases are then compared with the expected results.
• Report Test Results: This involves reporting the bugs that were found, specifically noting the test cases that caused the system to fail. Test reports are generated to document the test results. These reports list information such as the tester, the test plan, the module under test, and the test cases that produced unexpected results.

Automating the testing process

• Generate Test Cases Test cases can be generated automatically from the specification or source code. Automating test case generation is considered the most difficult job in automatic software testing.
• Generate Test Oracle A test oracle is used to determine the expected output for an input and can be derived from software specifications. Generating test oracles is considered difficult.
• Execute Test Cases This involves running the program under test with the generated test cases. This is relatively easy in automating software testing. A test driver reads test data, runs the program, and writes output for comparison. "Record – Playback" is a popular technique for regression testing.
• Compare Test Results The actual results from test execution are compared with the expected results determined by the test oracle. Comparing results is considered difficult.
• Generate Test Reports Reports are created for the test cases that cause the program to fail. Reports are prepared in designated formats. If a defect tracking system is linked, bugs should be automatically recorded. Generating test reports is considered easy.

Some tools available in the market support test automation, including tools for functional test automation, mobile testing, performance & load testing, static analysis, test design, and test management.

Difficulties of test automation

• Very difficult
  • Test case generation
• Difficult
  • Test oracle generation
  • Compare result
• Easy
  • Test execution
  • Test report
  • Test case management

Why testing is important?

• It is a very important activity used to verify the software.
• It is used to check if the software contains bugs that clients do not expect.
• Software testing is part of the overall validation process, which checks if the system meets users' requirements.
• The role of a software tester is considered important in software development.
• Lack of adequate testing can lead to significant financial loss and other negative consequences, as illustrated by the $100,000 keying error case.
• Industry feedback indicates a need for better software engineers and more training in software testing.

Configuration Management

• Configuration management is the development and use of standards and procedures for managing an evolving software system
• Configuration management is necessary for complex system

Configuration Management Activities

• Planning
• Change management
• Version control
• System building

Configuration Management plan

• Describes the standards and procedures that are used for configuration management
  • Define responsibilities
  • Define standards

Change management

What is change management?

• It is one of the configuration management activities.
• Analyzing the cost and the benefits of proposed changes.
• Approving those important changes.
• Tracking those changes

Ensures that all the changes are controlled.

How to do change management?

The change management process involves the following steps:

1. Request change.
2. Analyze change request: Check if the request is valid. If valid, assess how to change and assess the cost.
3. Record request (when submitted).
4. Submit request.
5. Asscept request: If the request is accepted, make the change, record it, and check its quality. If rejected, reject the request.
6. Quality OK: If the quality is OK, create a new version.

Version control

What is Version Control?

• It is one of the configuration management activities.
• Version control identifies and keeps track of versions of a system.
• Versions of a system can have different functionalities.
  • A system version is an instance of system
  • A system release is a version of a system that is delivered to customers.

How to do Version Control?

• Three techniques are used for version identification
  • Version numbering: Using explicit, unique version numbers (e.g., V1.0, V1.1, V2.0)
  • Attribute-based identification: Using the component name plus attributes.
  • Change-oriented identification: Similar to attribute-based identification, using the component name plus changes.

Release management, a related activity, involves

• Deciding when to release,
• Managing the process to create the release,
• Documenting the release.

A release may include various files like

• Configuration files,
• Data files,
• Installation programs
• Documentation.

System building

• The process of creating an executable system from the source code, often supported by build scripts, version management systems, compilers, and linkers.
• The process of creating an executable system.

Why is configuration management important?

• It is necessary for complex systems.
• It is very important in the software evolution and release process.
• It involves the development and use of standards and procedures for managing an evolving software system.

Cost Estimation

Measure Software size

• Size-related metrics,
  • Lines of Code (LOC or SLOC).
  • KLOC refers to Thousands of Lines of Code.
• Function-related metrics
  • Function Points (FPs)
  • Object Points (also called Application Points).

Function Point (FP)

• Function points (FPs) (# number of functions)

• Based on a combination of program characteristics

  • external inputs
  • external outputs;
  • user interactions;
  • external interfaces;
  • files

• The function point count is modified by complexity of the project

• Function point estimation is subjective.

  $$FPC=(numberofeachabovetype\times weight)$$

• AVC: AVerage KLOC per function point for a given language

  • 200 ~ 400 for assembly language
  • 2 ~ 4 for advanced language

$$LOC=AVC\times FPC$$

Object Point (OP)

• An alternative function-related measure to function points

• Also called application points.

• Object points not equal to number of object classes.

• The number of object points in a program is a weighted estimate of

  • the number of separate screens that are displayed;
  • the number of reports that are produced by the system;
  • the number of program modules that must be developed to supplement the database code

• Easier to be estimated than function points

• 4 ~ 50 object points / person-month

How to measure software development effort?

• Person-months (pm or PM) or man-month (mm).
• This measure represents the cost on engineers.

Cost Estimation Techniques

• Algorithmic cost modeling: Use the metric related to the project cost and the model that predicts the effort
• Expert judgement: Based on estimates from experts.
• Estimation by analogy: Compares the project with similar past projects.
• Parkinson’s law: Based on the available resource rather than the objectives.
• Pricing to win: Estimates the cost based on the investment available from the customer.

Algorithmic Cost Modeling

• A technique used in cost estimation,,.
• It uses a mathematical function to predict the effort required for a software project.
• The function uses product, project, and process attributes whose values are estimated by project managers.
• $$Effort=A\times Siz{e}^B\times M$$
  • $A$ is a constant specific to the organization.
  • $Size$ is a software size metric, such as KLOC (Thousands of Lines of Code),.
  • $B$ is an exponent, typically between 1 and 1.24, accounting for disproportionate effort in larger projects and influenced by factors like project novelty, development flexibility, risk management, and process maturity,,.
  • $M$ is a multiplier reflecting product, process and people attributes.
• Different algorithmic models are similar but use different values for A, B, and M.
• COCOMO (Constructive Cost Model), including COCOMO 2, is an example of an algorithmic cost model. COCOMO 2 includes sub-models like the Application composition model, Early design model, Reuse model, and Post-architecture model,.

COCOMO 2

• A COstructive COst MOdel (COCOMO).
• Proposed by Barry Boehm.
• An empirical model based on experiences.
• It is an updated version from 2000, based on a team spending over ten years residing in enterprises.
• It embraces different approaches to software development,
  • prototyping
  • spiral
  • component composition
• Produce detailed software estimates.
• Incorporates a set of sub-models whose application depends on the real development environments.

COCOMO 2 Sub-models

• Application composition model
• Early design model
• Reuse model
• Post-architecture model

Application Composition Model

• Supports prototyping projects and projects where there is extensive reuse
• Based on standard estimates of developer productivity in object-points/person-month
• Takes CASE tool use into account.

$$PM=\frac{(NOP\times (1-reuse))}{PROD}$$

• PM : Person-Month
• NOP: Number of Object Points (Application Points, NAP)
• PROD: Productivity (number of object points per person per month)
• reuse: Percentage of reused object points

PROD estimation

• Depend on the maturity and capability of process and developers

Early Design Model

• Used when requirements are available but design has not yet started.
• Estimates can be made after the requirements have been agreed.

$$PM=A\times Siz{e}^B\times M$$

• A = 2.94
• Size: KLOC
• B: 1 ~ 1.24
  • novelty of the project
  • development flexibility
  • risk management approaches
  • the process maturity

• $M=PERS\times RCPX\times RUSE\times PDIF\times PREX\times FCIL\times SCED$
  • RCPX: product reliability and complexity;
  • RUSE: the reuse required;
  • PDIF: platform difficulty;
  • PREX: personnel experience;
  • PERS: personnel capability;
  • SCED: required schedule;
  • FCIL: the team support facilities.

Reuse Model

• Used to compute the effort of integrating reusable components
• Two versions
  • Components reused without change
  • Components understood and integrated
    • The equivalent new source code must be estimated.

For code that does not need to be changed

• $$PM=\frac{(ASLOC\times AT)}{ATPROD}$$
• ASLOC: the total number of lines of adapted code
• AT: the percentage of adapted code that does not change (or can be automatically generated)
• ATPROD: the productivity of engineers in integrating this code.

For code that has to be understood and integrated

• $$ESLOC=ASLOC\times (1-AT)\times AAM$$
• ESLOC: The equivalent number of lines of new source code
• ASLOC: the total number of lines of adapted code
• AT: the percentage of code that does not need to be understood and integrated (e.g., does not change or can be automatically generated)
• AAM: the adaptation adjustment multiplier

Post-Architecture Model

Used once the system architecture has been designed and more information about the system is available

• $$PM=A\times Siz{e}^B\times M$$
• A = 2.94
• Size: KLOC
  • the number of lines of new code to be developed (NSLOC);
  • the equivalent number of lines of new code computed using the reuse model (ESLOC);
  • the number of lines of code that have to be modified according to requirements changes (MSLOC)
• B: Same as used in Early Design Model

Pricing to Win

• The project costs whatever the customer has to spend on it.
• Advantages
  • You get the contract
  • when detailed information is lacking it may be the only appropriate strategy
• Disadvantages
  • The probability that the customer gets the system he or she wants is small. Costs do not accurately reflect the work required
  • This approach may seem unethical and un-businesslike
  • The project cost is agreed on the basis of an outline proposal and the development is constrained by that cost.

Staffing and Calendar Time

• $$Effort=A\times SizeB\times M$$
• $$TDEV=3\times (PM)(0.33+0.2\times (B–1.01))$$